final report
Semantic Similarity in Radiology Reports via LLMs and NER
Pearson, Beth, Adnan, Ahmed, Abdallah, Zahraa S.
Radiology report evaluation is a crucial part of radiologists' training and plays a key role in ensuring diagnostic accuracy. As part of the standard reporting workflow, a junior radiologist typically prepares a preliminary report, which is then reviewed and edited by a senior radiologist to produce the final report. Identifying semantic differences between preliminary and final reports is essential for junior doctors, both as a training tool and to help uncover gaps in clinical knowledge. While AI in radiology is a rapidly growing field, the application of large language models (LLMs) remains challenging due to the need for specialised domain knowledge. In this paper, we explore the ability of LLMs to provide explainable and accurate comparisons of reports in the radiology domain. We begin by comparing the performance of several LLMs in comparing radiology reports. We then assess a more traditional approach based on Named-Entity-Recognition (NER). However, both approaches exhibit limitations in delivering accurate feedback on semantic similarity. To address this, we propose Llama-EntScore, a semantic similarity scoring method using a combination of Llama 3.1 and NER with tunable weights to emphasise or de-emphasise specific types of differences. Our approach generates a quantitative similarity score for tracking progress and also gives an interpretation of the score that aims to offer valuable guidance in reviewing and refining their reporting. We find our method achieves 67% exact-match accuracy and 93% accuracy within +/- 1 when compared to radiologist-provided ground truth scores - outperforming both LLMs and NER used independently. Code is available at: https://github.com/otmive/llama_reports
Amazon, Google and Meta are 'pillaging culture, data and creativity' to train AI, Australian inquiry finds
Tech companies Amazon, Google and Meta have been criticised by a Senate select committee inquiry for being especially vague over how they used Australian data to train their powerful artificial intelligence products. Labor senator Tony Sheldon, the inquiry's chair, was frustrated by the multinationals' refusal to answer direct questions about their use of Australians' private and personal information. "Watching Amazon, Meta, and Google dodge questions during the hearings was like sitting through a cheap magic trick โ plenty of hand-waving, a puff of smoke, and nothing to show for it in the end," Sheldon said in a statement, after releasing the final report of the inquiry on Tuesday. He called the tech companies "pirates" that were "pillaging our culture, data, and creativity for their gain while leaving Australians empty-handed." The report found some general-purpose AI models โ such as OpenAI's GPT, Meta's Llama and Google's Gemini โ should automatically default to a "high risk" category, and be subjected to mandated transparency and accountability requirements.
'Eugenics on steroids': the toxic and contested legacy of Oxford's Future of Humanity Institute
Two weeks ago it was quietly announced that the Future of Humanity Institute, the renowned multidisciplinary research centre in Oxford, no longer had a future. It shut down without warning on 16 April. Initially there was just a brief statement on its website stating it had closed and that its research may continue elsewhere within and outside the university. The institute, which was dedicated to studying existential risks to humanity, was founded in 2005 by the Swedish-born philosopher Nick Bostrom and quickly made a name for itself beyond academic circles โ particularly in Silicon Valley, where a number of tech billionaires sang its praises and provided financial support. Bostrom is perhaps best known for his bestselling 2014 book Superintelligence, which warned of the existential dangers of artificial intelligence, but he also gained widespread recognition for his 2003 academic paper "Are You Living in a Computer Simulation?".
Hybrid LLM/Rule-based Approaches to Business Insights Generation from Structured Data
Vertsel, Aliaksei, Rumiantsau, Mikhail
In the field of business data analysis, the ability to extract actionable insights from vast and varied datasets is essential for informed decision-making and maintaining a competitive edge. Traditional rule-based systems, while reliable, often fall short when faced with the complexity and dynamism of modern business data. Conversely, Artificial Intelligence (AI) models, particularly Large Language Models (LLMs), offer significant potential in pattern recognition and predictive analytics but can lack the precision necessary for specific business applications. This paper explores the efficacy of hybrid approaches that integrate the robustness of rule-based systems with the adaptive power of LLMs in generating actionable business insights.
NSF-led National Artificial Intelligence Research Resource Task Force Releases Final Report
Today, the National Artificial Intelligence Research Resource (NAIRR) Task Force released its final report, a roadmap for standing up a national research infrastructure that would democratize access to the resources essential to artificial intelligence (AI) research and development. Established by the National AI Initiative Act of 2020, the NAIRR Task Force is a federal advisory committee. Co-chaired by the U.S. National Science Foundation and the White House Office of Science and Technology Policy, the Task Force has equal representation from government, academia, and private organizations. Following its launch in June 2021, the Task Force embarked on a rigorous, open process that culminated in this final report. This process included 11 public meetings and two formal requests for information to gather public input.
Federal R&D investments serve as foundation for US becoming AI-ready
The National Security Commission on Artificial Intelligence, in its final report to Congress and the Biden administration last year, warned artificial intelligence will soon become "weapons of first resort in future conflicts." That warning, as well as the commission's recommendation for the federal government to increase spending on basic research and development, remains urgent for the U.S. to remain AI-ready in the coming years, even though the commission no longer remains. The commission disbanded in October 2021, but many of its leading experts have shifted to a private-sector entity, the Special Competitive Studies Project (SCSP). The name stems from the Rockefeller Special Studies Project, launched in 1956 by Nelson Rockefeller and Henry Kissinger following the Soviet Union's launch of the satellite Sputnik. SCSP chief executive officer Ylli Bajraktari, NSCAI's former executive director, said Rockefeller and Kissinger saw their project as a way for the U.S. to further define its national objectives when it came to defense, security and foreign policy. "This is not the first time that we're seeing technology playing a critical role in great power competition," Bajraktari said.
Artificial intelligence: the EU needs to act as a global standard-setter
The adopted text says that the public debate on the use of artificial intelligence (AI) should focus on this technology's enormous potential to complement humans. The text warns that the EU has fallen behind in the global race for tech leadership. As a result, there is a risk that standards will be developed elsewhere in the future, often by non-democratic actors, while the EU needs to act as a global standard-setter in AI. MEPs identified policy options that could unlock AI's potential in health, the environment and climate change, to help combat pandemics and global hunger, as well as enhancing people's quality of life through personalised medicine. AI, if combined with the necessary support infrastructure, education and training, can increase capital and labour productivity, innovation, sustainable growth and job creation, they add.
UC creates recommendations for responsible use of artificial intelligence
The University of California has created recommendations to create a path toward the responsible use of artificial intelligence in future UC endeavors. UC's increasing dependence on the use of AI has increased its overall productivity as an institution, according to the UC Office of the President, or UCOP. However, with the implementation of AI, there is also potential for problems to arise. To combat this, former UC President Janet Napolitano and current president Michael Drake created the Presidential Working Group on Artificial Intelligence, or the Working Group, in August 2020. The Working Group's final report noted that the group consists of 32 faculty and staff from all 10 UC campuses and an additional number of representatives from UC Legal and the Office of Ethics, Compliance and Audit Services, among other groups.
America's 'Smart City' Didn't Get Much Smarter
In 2016, Columbus, Ohio, beat out 77 other small and midsize US cities for a pot of $50 million that was meant to reshape its future. The Department of Transportation's Smart City Challenge was the first competition of its kind, conceived as a down payment to jump-start one city's adaptation to the new technologies that were suddenly everywhere. Ride-hail companies like Uber and Lyft were ascendant, car-sharing companies like Car2Go were raising their national profile, and autonomous vehicles seemed to be right around the corner. "Our proposed approach is revolutionary," the city wrote in its winning grant proposal, which pledged to focus on projects to help the city's most underserved neighborhoods. It laid out plans to experiment with Wi-Fi-enabled kiosks to help residents plan trips, apps to pay bus and ride-hail fares and find parking spots, autonomous shuttles, and sensor-connected trucks.
Artificial intelligence and the future of national security
Artificial intelligence is a "world-altering" technology that represents "the most powerful tools in generations for expanding knowledge, increasing prosperity and enriching the human experience" and will be a source of enormous power for the companies and countries that harness them, according to the recently released Final Report of the National Security Commission on Artificial Intelligence. This is not hyperbole or a fantastical version of AI's potential impact. This is the assessment of a group of leading technologists and national security professionals charged with offering recommendations to Congress on how to ensure American leadership in AI for national security and defense. Concerningly, the group concluded that the U.S. is not currently prepared to defend American interests or compete in the era of AI. The NSCAI was chartered by Congress in August 2018 to review AI and related technologies and make recommendations to address U.S. national security and defense needs.